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REMARKS 

In the Office Action, claims 1,3-21, 23-25, 27-46, 48-67 and 69 were rejected 
under 35 U.S.C. § 103(a) as allegedly obvious over U.S. Patent No. 5,684,715 to Palmer 
(hereinafter "Palmer") in view of "MPEG-7 Requirements." Claims 22, 47, 68 and 70-75 
were rejected as allegedly obvious under 35 U.S.C. § 103(a) in view of these same 
references further viewed in combination with U.S. Patent No. 6,466,940 to Mills 
(hereinafter "Mills"). Applicants respectfully traverse the rejections of record, and further 
submit that claims 1-75 as amended are in condition for allowance. 

Claims 2, 24 and 26 were objected to as allegedly containing the same limitation 
("extraction processing") twice. However, as to claims 2 and 26, the claimed phrases 
"video object extraction processing" and "object hierarchy construction and extraction 
processing" are two separate and distinct steps. Accordingly, Applicants respectfully 
submit that these claims are correct in their present form. Applicants have amended claim 
24 to correct the identified informality. Applicants respectfully submit that these claims 
are now in condition for allowance. 

Additionally, the Examiner has objected to the drawings as containing informalities. 
Specifically, the Examiner has objected to Figure 8 parts 231, 241, 251, 261, 271 and 281 
for reading "MPEC" as opposed to "MPEG." Applicants provide herewith a replacement 
sheet for Figure 8 with the suggested corrections. 

Finally, the Examiner has objected to the specification for not containing an 
Abstract of the Disclosure, as required by 37 C.F.R. 1.72(b), and for containing several 
informalities. Applicants provide herewith an Abstract on a separate sheet, as required by 
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the rules, and have corrected the cited informalities by amendment to the Specification and 
correction to Figure 13. 

Finally, Applicants thank the Examiner for identifying allowable subject matter in 
claims 2 and 26. 

Rejections under 35 U.S.C. § 103(a) over Palmer in view of "MPEG-7 
Requirements" and further in view of Mills 

Claims 1, 3-21, 23-25, 27-46, 48-67 and 69 were rejected under 35 U.S.C. § 103(a) 
as allegedly obvious over U.S. Patent No. 5,684,715 to Palmer (hereinafter "Palmer") in 
view of "MPEG-7 Requirements." 

Independent claim 1 is directed to a system for generating a description record from 

video information, comprising, inter alia: 

a computer processor coupled to said at least one video input interface for receiving 
said video information therefrom, processing said video information by performing 
video object extraction processing to generate video object descriptions from said 
video information, processing said generated video object descriptions by object 
hierarchy construction and extraction processing to generate video object hierarchy 
descriptions, and processing said generated video object descriptions by entity 
relation graph descriptions, wherein at least one description record including said 
video object descriptions, said video object hierarchy descriptions and siad entity 
relation graph descriptions is generated to represent content embedded within said 
video information; 

Independent claim 25 includes similar limitations in the context of a method claim, and 
independent claim 49 includes similar limitations in the context of a computer readable 
media claim. 
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The Examiner rejected independent claims 1 , 25 and 49 partly based on "MPEG-7 
Requirements," as discussed on pages 4-5 of the Office Action: 

"Palmer does not disclose a system where processing said generated video object 
descriptions by object hierarchy construction and extraction processing to generate 
video object hierarchy descriptions, and wherein at least one description record 
including said video object descriptions, said video object hierarchy descriptions 
and said entity relation graph descriptions is generated to represent content 
embedded within said video information. 

MPEG-7 Requirements disclose a system where processing said generated video 
object descriptions by object hierarchy construction and extraction processing to 
generate video object hierarchy descriptions (page 5, figure 1; page 10, lines 4-6), 
and wherein at least one description record including said video object descriptions, 
said video object hierarchy descriptions and said entity relation graph descriptions 
is generated to represent content embedded within said video information (page 6, 
figure 3). 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art to use a hierarchy of descriptions for the video descriptors disclosed in 
Palmer as taught by MPEG-7 Requirements. The motivation for doing this would 
have been to allow for queries to be processed more efficiently (page 9, bullet 7)." 

However, as an initial matter, these rejections under 35 U.S.C. § 103(a) are improper at 

least because the cited "MPEG-7 Requirements" is not an enabling disclosure and 

therefore cannot properly be cited as a prior art reference. As is clear from the contents of 

the MPEG-7 Requirements document, the document was intended to set forth exactly that 

- "requirements" ~ for a system which was not yet known and/or implemented in the prior 

art. If anything, the mere existence of this document, released by the MPEG-7 

standards-development group just one month prior to the priority date of the present 

application, establishes that no working system for meeting these requirements was yet 

known in the art, and emphasizes the inventiveness of the present invention. 
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Further support can be found in the remaining two MPEG-7 documents cited by the 
Examiner in connection with the Office Action, but not relied upon, including: 

http://www.tnt.uni-hannover.de/project/mpeg/audio/public/mpeg7/w2464.html 

which states: 

"MPEG-7 will standardize the description of Audio-Visual Content, but will not 
standardize the algorithms for extracting the descriptions or searching / filtering 
using these descriptions. For details on the background and goals of MPEG-7 
please refer to MPEG-7: Context and Objectives document [2], 

MPEG-7 will address applications that can be stored (on-line or off-line) or 
streamed (e.g. broadcast, push models on the Internet), and can operate in both 
real-time and non real-time environments. A 'real-time environment' means that 
information is associated with the content while it is being captured." 

The future tense used in these applications further supports that no working system was 
developed as of October 1998, and further, that the cited MPEG-7 documents are clearly 
not enalbing to one of ordinary skill in the art such that they may properly be cited as prior 
art by the Patent Office. For at least these reasons, Applicants respectfully submit that the 
rejections of record are improper, and further that the claims as amended should be 
allowed. 

A further question may even be raised as to whether this MPEG-7 Requirements 
document was published as of the date indicated on its first page, or whether this was 
simply the date the document was created and distributed within the MPEG Working 
Group standards body on a confidential basis prior to any public release of the document. 
For example, as noted at the MPEG Working Group website: 
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"As a result of this working method MPEG manages some 500 documents 
at each meeting. About 300 are input documents from members and about 
200 are output documents produced by the committee. These documents 
are restricted to MPEG memebers. From time to time, however, MPEG 
decides to post publicly some of its output documents. These are typically 
calls for proposals, general descriptions of standards, approved or under 
development, the text of standards under ballot etc." 

http://www.chiariglione.org/mpeg/documents.htm 

For at least these reasons, Applicants respectfully submit that the rejections of 
record should be withdrawn. 

Additionally, Palmer is directed to an interactive video system in which an operator 
is able to select an object moving in a video sequence and by which the system is notified 
which object was selected so as to take appropriate action. (Palmer, col. 1, Ins. 47-50). 
"More particularly, the invention concerns a generator for generator dynamic object 
descriptors which correspond to objects moving in a video sequence, and an event 
interpreter for using te object descriptors in connection with playback of the video 
sequence so as to detect and act upon operator selection of the objects." (Palmer, col. 1 , Ins. 
10-16). 

As an initial matter, Palmer is directed to a different function and purpose than the 
present invention. While the system of Palmer is intended to allow more interactivity with 
a video sequence, the present invention is directed to a system for generating a description 
record from multimedia information in order to, e.g., allow for searching and browsing of 
video content. (See, e.g., Specification, p. 2, Ins. 9-17). 

In the Office Action, the Examiner contends that Palmer discloses performing 
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video object extraction processing to generate video object descriptions from said video 
information. (Office Action, p. 4, citing Palmer at col 1, Ins. 10-15). However, the 
claimed "video object descriptions" are not disclosed or suggested by Palmer. Rather, 
Palmer discloses "video object descriptors" which are illustrated in Fig. 3, element 60, and 
described in more detail at col. 4, Ins. 51-67. The video object descriptors of Palmer 
include information ID, shape, size, layering, duration, action, etc. -- i.e., the sorts of object 
information that would be relevant and useful to a system such as Palmer which 
manipulates objects in video sequences. However, this is not the same information that 
makes up the "video object descriptions" of the claimed invention. The video object 
descriptions of the claimed invention are intended to provide information about the video 
sequences such that the contents of a database of video sequences may be searchable, 
categorized, and the like. By way of example, these descriptions may include semantic 
information about the contents of the video such as "who, what object, what action, why, 
when, where," (Specification, p. 13). The end result is that the present invention describes 
the video content such that databases containing the video content may be later searched 
using queries. Palmer does not disclose or suggest such functionality (and again, does not 
need to, since this is not the goal of Palmer). 

Additionally, the Examiner contends that Palmer discloses "processing said 
generated video object descriptions by entity relation graph generation processing to 
generate entity relation graph descriptions." (Office Action, p. 4, citing Palmer at col. 7, 
Ins. 46-50). As defined on pp. 19-20 of the Specification of the present application, an 
entity relation graph "is a graph of one or more entity nodes 42 and the relationships among 
them." The cited portion of Palmer does not teach "entity relation graph descriptions" at 
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all. (The cited portion of Palmer, at col. 7, Ins. 46-50, instead refers to video object 
descriptors stored in a file, and extracting from those descriptors a "list 87 of active objects, 
and an action map 89 which includes a list of actions corresponding to all valid objects in 
the list 87 of active objects.") In fact, as described in other portions of Palmer, the system 
of Palmer discloses generating a "frame-sequential file of video object descriptors from 
successive frames of video information." (Palmer, col. 2, Ins. 36-49). Palmer provides no 
additional description regarding the ways in which Palmer's data is 
created/organized/maintained, other than it is stored in a file in some sort of list. 

For at least these reasons, Applicants respectfully submit that the combination of 
Palmer and the MPEG-7 Requirements fail to disclose or suggest several elements of the 
claimed invention. 

Furthermore, Applicants respectfully assert that this combination of Palmer and the 
MPEG-7 Requirements document under 35 U.S.C. § 103 is improper, and was apparently 
improperly made only with the benefit of hindsight in view of the present invention. As the 
Court of Appeals for the Federal Circuit has held: 

It has not been shown that a person of ordinary skill, seeking to solve a 
problem of fastening a hose clamp, would reasonably be expected or 
motivated to look to fasteners for garments. The combination of elements 
from non-analogous sources, in a manner that reconstructs the applicant's 
invention only with the benefit of hindsight, is insufficient to present a 
prima facie case of obviousness. There must be some reason, suggestion, or 
motivation found in the prior art whereby a person of ordinary skill in the 
field of the invention would make the combination. That knowledge can 
not come from the applicant's invention itself. 
In re Oetiker, 24 U.S.P.Q.2d 1443, 1447, 977 F.2d 1443, 1447 (Fed. Cir. 1992). 
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There is no "reason, suggestion, or motivation" in the prior art such that one of 
ordinary skill in the art would make the combinations which form the basis of the rejections 
under 35 U.S.C. § 103(a) in the Office Action. This combination of elements between 
non-analogous sources — i.e., a system for generating descriptions of video content for the 
purposes of searching/indexing, and an interactive system for allowing an operator to 
select video objects in interactive video — is apparently improperly made only with the 
benefit of hindsight in view of the present application. In much the same way that one 
seeking to solve a problem of "fastening a hose clamp" would not "reasonably be expected 
or motivated to look to fasteners for garments" for a solution, it is equally unlikely that one 
seeking to solve a problem in video content searching/indexing would look to an 
interactive video system, and vice-versa. Accordingly, without the benefit of hindsight, 
one of ordinary skill in the art would not look to combine this non-analogous art. For at 
least this additional reason, Applicants respectfully submit that the claims as amended 
should be allowed. 
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CONCLUSION 

In view of the foregoing remarks, favorable consideration and allowance of claims 
1-75 as amended are respectfully solicited. In the event that the application is not deemed 
in condition for allowance, the examiner is invited to contact the undersigned in an effort to 
advance the prosecution of this application. 

Respectfully submitted, 

Paul A. Ragusa 
PTO Reg. No. 38,587 

Robert L. Maier 
PTO Reg. No. 54,291 

Attorneys for Applicants 
(212) 408-2538 

BAKER BOTTS L.L.P. 
30 Rockefeller Plaza 
New York, NY 10112 
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